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1  MOTIVATION 

The  motivation  behind  our  discussion  and  analysis  focused  on  understanding  the  realities  of  current 
political,  strategic,  operational  and  tactical  environments.  Current  operations  involve  an  increasingly  rapid 
tempo,  involving  short  decision  cycles  and  high  stress.  There  is  an  increased  link  between  the  units  in 
which  political  considerations  affect  strategy,  operations  and  tactics  on  a  short  term  basis.  While  there  are 
fewer  tactical  units,  these  units  have  increased  fire  power.  Hence  the  activities  of  an  individual  tactical 
unit  can  have  a  greater  impact  at  tactical,  strategic  and  political  arenas  (in  a  negative  and  positive  way). 
This  is  made  possible  due  to  increased  communications  capacity  (both  bandwidth  and  connectivity). 
Thus,  network  centricity  is  a  double-edged  sword. 


2  GROUP  PROCESS 

The  process  that  we  used  for  this  group  involved  first  defining  the  operational  needs  from  the  viewpoint  of 
a  senior  command  officer  versus  an  analyst.  We  wanted  to  understand  key  design  considerations  and  basic 
visualization  concepts  that  would  work  in  this  fast  paced,  high  stress  environment.  We  also  discussed  what 
reliability  and  uncertainty  means  from  the  viewpoint  of  the  command  officer.  We  then  discussed  what  we 
viewed  as  technology  gaps  in  achieving  the  necessary  situational  awareness  specifically  as  it  pertains  to 
addressing  the  reliability  of  sources  and  uncertainty  of  the  data.  We  concluded  by  identifying  both  short 
and  long  term  recommendations  for  further  investigation  and  experimentation. 


3  CHALLENGES 

The  operational  realities  previously  identified  imply  a  number  of  challenges  for  visualization.  First,  we 
must  recognize  the  need  to  support  all  levels:  strategic,  operational  and  tactical.  One  of  the  key  challenges 
is  to  make  a  common  operating  picture  (COP)  that  is  consistent,  repeatable  (under  identical  query 
conditions),  and  customizable  depending  of  the  focus  of  attention.  Another  key  challenge  is  how  to 
represent  uncertainty  in  a  consistent  “repeatable”  manner  such  that  it  can  be  used  to  make  decisions.  The 
last  key  challenge  we  discussed  is  how  to  measure  the  reliability  of  a  given  source.  Given  that  past 
performance  may  not  be  a  good  enough  indicator  for  future  performance,  understanding  (and  thus, 
visualizing)  reliability  is  a  difficult  challenge. 


4  DESIGN  IMPLICATIONS 

Some  basic  design  considerations  are  to  keep  the  information  displays  simple  to  avoid  information 
overload.  In  general,  analogue  displays  are  preferred  in  lieu  of  digital  displays  (provides  a  “visual 
snapshot”  of  what  is  normal  without  having  to  “read”  any  data).  It  is  also  important  that  the  user  has 
adequate  control  to  adjust  the  level  of  detail  and  the  focus  of  attention.  However,  too  many  controls  will 
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allow  too  many  options  and  complicate  the  process.  While  many  attempts  have  already  occurred,  it  is  also 
important  that  symbology  achieve  a  better  level  of  consistency.  Finally,  it  is  critical  that  the  design  has  full 
user  feedback  validation  across  a  statistically  significant  cross  section.  Assuming  new  processes  or 
procedures  may  result  due  to  capabilities  improvement,  it  is  important  to  ensure  that  the  change  be 
handled  in  such  a  manner  to  permit  a  smooth  transition  (including  “doing  things  the  old  way”). 


5  RELIABILITY  AND  UNCERTAINTY 

Before  we  can  begin  to  visualize  concepts  such  as  reliability  and  uncertainty,  we  need  to  come  to  a  firm 
understanding  of  what  we  mean  when  using  these  terms.  One  view  is  to  think  of  reliability  as  a  measure  of 
the  information  source  which  may  vary  as  a  function  of  the  task.  It  may  be  difficult  to  “quantify”  or  define 
metrics  (or  confidence  ratings)  in  a  consistent  manner.  To  be  meaningful,  however,  some  degree  of 
consistency  and  repeatable  is  critical.  Uncertainty,  on  the  other  hand,  is  a  measure  of  the  information.  It 
can  vary  as  a  function  of  time  as  in  the  case  of  a  reported  aircraft  that  is  “lost”  from  radar.  Every  second 
that  it  is  lost,  the  uncertainty  of  the  aircraft  location  increases  from  a  geospatial  viewpoint.  On  the  other 
hand,  object  identification  can  also  undergo  descriptions  of  uncertainty  by  using  such  terms  as  possible, 
probable,  or  confirmed.  If  these  terms  are  defined  using  consistent  (standardized)  criteria  such  that  the 
resulting  uncertainty  rating  is  repeatable,  then  visualization  techniques  are  more  readily  conceptualized. 


6  INFORMATION  LAYER  CONCEPT 

The  illustration  below  is  shows  how  information  layers  can  be  used  to  both  convey  what  we  know 
(information)  and  what  we  don’t  know  (lack  of  information).  The  information  that  we  don’t  know  can  then 
be  used  as  a  rough  measure  of  the  uncertainty  associated  with  that  information  layer.  The  information  in 
these  layers,  starting  from  the  foundation  terrain/map/image  layers  to  the  intelligence  overlay,  is  used  to 
create  a  series  of  assessment  overlays  such  as  a  threat  assessment  or  air  capacity  assessment.  As  an 
example,  radar  coverage  from  a  surface-to-air  missile  (SAM)  would  “paint”  a  3-D  “red”  (threat)  dome  that 
may  be  partially  obscured  by  terrain  elevation  (nearby  hill).  Uncertainty  at  the  fringe  of  the  radar  coverage 
could  be  colour  shaded  yellow  or  orange  as  a  simple  indicator  that  there  is  uncertainty  in  the  potential 
danger  area  of  these  SAMs.  These  assessments  can  also  be  done  within  the  context  of  an  operational 
mission  such  as  suppression  of  enemy  air  defence  (SEAD)  where  we  can  (automatically)  custom  adjust  the 
sensor  coverage  information  to  determine  what  we  don’t  know  (i.e.,  scanning  for  indicators  that  can  shoot 
down  an  aircraft)  while  ignoring  other  coverage  shortfalls  that  are  not  pertinent  to  this  assessment.  In 
addition  to  the  concept  of  information  layers  and  the  threat  assessment  overlays,  it  is  also  important  to  be 
able  to  adjust  the  field  of  view  (FOV  or  God’s  Eye  Zoom)  and  customize  the  “decluttering”  of 
information.  The  “zoom”  adjusts  the  geo-spatial  content  while  a  time  “slider”  permits  reviewing  all 
available  information  (within  this  FOV).  The  next  figure  illustrates  a  further  refinement  in  how  one  can 
declutter  and  customize  the  FOV. 
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Information  Layer  Concept 


Threat  Assessment 
Intelligence 
Sensor  Coverage 


Military  Installation 

Satellite  Image  / 
Weather  Data 

Reference  (map 
data) 


Risk  Assessment 


Working  from  the  previous  guidelines,  a  recently  taken  image  is  used  as  a  foundation  map  with  map 
features  overlaid  (such  as  roads  and  borders).  Within  the  “zoomed”  FOV  and  a  time  period  (set  by  the  user 
and  defaulted  to  “current  time  to  X  hours  prior”),  a  list  of  available  sensors  that  were  available  in  that 
given  spatial  area  and  time  frame  is  displayed.  The  user  can  “turn  on”  the  display  for  the  list  of  available 
sensors  to  overlay  on  the  map/image.  The  sensor  timeline  also  shows  the  time  period  that  the  sensor  was 
active  and  operational.  This  collection  of  information  can  then  be  used  to  determine  what  we  don’t  know 
and  provide  an  indication  of  uncertainty  (with  regards  to  coverage)  and  also  the  reliability  of  the  threat 
assessment.  In  other  words,  a  threat  assessment  that  might  indicates  “little  or  no  threat”  with  a  significant 
lack  of  coverage  is  not  reliable  and  should  not  arrive  at  that  conclusion. 
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Conceptual  Example 
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Other  examples  of  limited  operational  and  prototype  systems  that  work  with  information  layers  previously 
described  are  as  follows: 

•  Common  Operating  Decision  System  (CODS)  -  video  demonstration  available 

•  Distributed  Data  Fusion  Simulator  (DDFS) 

•  Multi-sensor  Integration  and  Intelligence  Analysis  (MSIIA) 


7  TECHNOLOGY  ASSESSMENTS/GAPS 

Prior  experimentation  has  shown  that  search  related  tasks  on  display  devices  can  use  nine  (9)  mega-pixels 
of  display  resolution  (at  the  proper  distance  and  pixel  size)  when  the  information  display  is  viewed 
statically.  As  the  visual  task  gets  more  complicated  with  multiple  points  of  interest,  there  is  strong 
indication  that  more  pixel  display  area  can  benefit  the  users  by  shortening  the  decision  process  (due  to  not 
having  to  scroll  or  retrieve  readily  available  information).  Component  technology  to  support  this  is 
available  but  further  development  may  still  be  needed  to  determine  how  best  to  integrate  the  fusion  of 
information  within  the  visually  rich  display  area.  Another  shortfall  in  better  addressing  uncertainty  and 
reliability  is  the  need  for  better  representation  of  uncertainty  and  reliability.  These  representations  need  to 
be  well  understood  and  repeatable  under  the  same  conditions.  Once  these  representations  have  been 
established,  progress  to  creating  better  visualization  techniques  can  be  made. 
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8  RECOMMENDATIONS 

The  short  term  recommendations  are  as  follows: 

•  Initiate  operational  studies  and  analyses  of  visualization  needs  from  the  analyst’s  and 
commander’s  viewpoints 

•  Develop  visualization  concepts  and  prototypes,  defining  what  uncertainty  and  reliability  conveys 

•  Conduct  experiments  with  representations  of  uncertainty  and  reliability. 


One  “low  hanging  fruit”  is  to  use  higher  resolution  display  technology  or  an  appropriately  matrixed  array 
of  high  resolution  displays  oriented  in  such  as  manner  as  to  maximize  the  information  availability  without 
“overloading”  the  user.  Further  research  may  improve  how  displays  of  this  caliber  can  be  organized  to 
maximize  the  information  output  without  creating  an  overload  situation. 

The  long  term  recommendations  are  as  follows: 

•  Development  of  consistent  techniques  for  determining  uncertainty  and  reliability 

•  Development  of  intuitive  techniques  for  visualizing  uncertainty  and  reliability 

•  Experimentation  with  3-D  visualization 

•  Deployment  of  real  systems. 
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Motivation  -Current  Reality 


■  Operational  Tempo 

□  Short  Decision  Time 

□  High  Stress 

■  Increased  Link  Between  Political,  Strategic 
and  Tactical  Units 

■  Increased  Tactical  “Fire  Power”  (Increased 
Tactical  --  Strategic  Impact) 

■  Increased  Communication  Capacity 


Group  Process 

■  Definition  phase 

□  Operational  needs 

■  Development  /  design  guidelines 

■  Visualization  concept 

■  Meaning  of  reliability  and  uncertainty 

■  Technology  assessment/gaps 

■  Short  and  long  term  recommendations 
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Challenges  for  Visualization 


■  Effective  visualization  of  strategic, 
operational  and  tactical  situations 

■  Representation  and  visualization  of  land 
forces 

■  How  to  represent  uncertainty 

□  Representation  of  what  we  don’t  know 
(missing  versus  negative  information) 

□  Representation  of  uncertainty  versus 
reliability 


Design  Implications 


■  Keep  it  simple/avoid  information  overload 

■  Analog  versus  digital  representations 

■  User  control 

□  Level  of  detail 

□  Focus  of  attention 

■  Standardization  of  symbology 

■  Design  validation  by  users 

□  Carefully  planned  change  management/transition 
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ORGANIZATION 

Reliability  and  Uncertainty  (one  view) 


Reliability 

□  Is  a  measure  of  the  information  source 

□  Can  vary  as  a  function  of  the  task 

□  May  be  difficult  to  “quantify”  consistently 
(define  metrics  or  “confidence”  rating) 


Object  is  at 
location  x,y 


reliable 


Object  will  launch 
at  time  z 

unreliable  ? 


■  Uncertainty 

□  Is  a  measure  of  the  information 

□  Can  vary  as  a  function  of  time 

□  Requires  standardization  to  “quantify”  consistently 
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Conceptual  Example 
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Air  Operations  Center 
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Examples  of  Information  Fusion 
Systems 

■  Common  Operating  Decision  System 
(CODS) 

■  Distributed  Data  Fusion  Simulator  (DDFS) 

■  Multi-sensor  Integration  &  Intelligence 
Analysis  (MSIIA)  -  conceptual  example 
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DDFS  -  Example 
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Technology  Assessment/Gaps 

■  Resolution  of  Display  Device 

■  Better  representation  of  uncertainty  and 
reliability 

■  Better  visualization  techniques  for 
uncertainty  and  reliability 

■  Repeatable  transformations 


Recommendations 


■  Short  Term 

□  Increased  resolution  displays  (satellite  images 
superimposed  over  maps)  appropriately  integrated 
Operational  studies  and  analyses  of  visualization  needs 

□  Concept  development  and  prototyping 
Experiments  with  representations  of  uncertainty  and 
reliability 

■  Long  Term 

Development  of  consistent  techniques  for  determining 
uncertainty  and  reliability 

□  Techniques  for  intuitive  visualization  of  uncertainty  and 
reliability 

Experimentation  with  3-D  visualization 
Deployment  of  real  systems 
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Keeping  it  Simple 
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